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Abstract 

With the development of e-learning, the quality of web-based courses attracts extensive interest. This paper draws 
upon the results conducted amongst students enrolled in an online language course at a northern Chinese university. 
The design of the course aims to create the learner-centered environment: personalized learning environment, 
peer-learning environment, and a new School-to-Work model etc. By using a special framing, the three main factors 
are closely integrated in one online course design. Based on the data collected and analysis, the study indicates 
establishing an effective and practical assessment index system will help maintain the high quality of online learning. 

Keywords: Web-based course. Personalized learning environment. Peer-learning environment, A new 
School-to-Work model. Effective formative assessment index system 

1. Introduction 

In recent years, education field has moved rapidly towards integrating technology or computer in the process of 
teaching and learning. This is because computer has a great potential for enhance teaching and learning outcomes 
(Fisher, 2010; Suchanska & Keczkowska, 2007). Online learning as defined by Chang and Fisher (2003) is a system 
and process that connects learners with the materials and information that distributed online. Rigid timetables, 
classroom-based teaching, theoretical concepts—these were once the hallmarks of higher education programs. Today 
they are beginning to fade as more and more students opt for the flexibility, accessibility, and career-skills focus of 
online and continuing education. Online education is opening up opportunities for a growing market of 
non-traditional learners. In fact, the number of non-traditional undergraduate students (over the age of twenty-five) is 
expanding faster than the traditional student group. 

But as with anything new and potentially disruptive, skeptics abound. There are many controversial issues related to 
the effectiveness of online instruction. One claim that online instruction lacks the ability to satisfy the diverse 
learning needs of online learners. In other words, unfortunately, many educational websites do not employ principles 
of effective learning (Cook& Dupras, 2004). Others argue that the whole process of online leaning is hard to be 
tracked down and evaluated, so the quality of instruction cannot be guaranteed. There is still strong challenge from 
those who pay more attention to practice. They point out that there is a great gap between online learning and real job 
market. Since users are moving toward online learning, we should consider the following questions. How to attract 
the users individually? How to give them more motivation, independence and self-directed learning? How to design 
a website to match the individuals? How to convey information with different environments to a diverse set of 
personal types? How different individual learners interact with the web-based instruction? What kind of individual 
differences that our users have (Martinez, 2001; Chen& Paul, 2003)? Furthermore, how can online courses compete 
with lecture-style courses? What are the biggest draws of online learning to the job market? 

Although many researches have been made in this field, the problem deserves new research. All previous research 
results are rarely integrated in one online course design. And current online course design still can not answer how to 
enhance the connection between online learning and the future job market. Therefore based on such online course 
design, the assessment index system cannot guarantee the high quality of online learning. More innovative work on 
the assessment index is needed. 

This study was motivated by the need to develop an effective online assessment index system. Based on a study 
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report on an Interpretation online course in a Chinese university, the author initiates a unique assessment index 
system for online learning which motivates both instructors’ and learners’ engagement and cultivate the learners’ 
practical ability to get the hands-on skills they need to be able to function on the job and become the successful job 
candidates in today’s competitive job market. 

2. Literature Review 

According to Attwell (2007) Personal Learning Environment is an environment that constructed by individual. 
Individuals are responsible for their own learning more effectively and take a larger stake in the ownership of the 
content. Riecken (2000) stated that personalization helps teacher to build a meaningful one to one relationship with 
students, by understanding their needs. The personalized environment also helps students to reach a goal because 
their individuals’ need is well addresses in a given context. Motivation theorists argue that the reason individuals 
perform actions is not only because of external interests but for their intrinsic needs such as happiness, enjoyment, 
and curiosity (Malone, 1981; Davis, Bagozzi & Warshaw, 1992). Researchers have found that people will spend 
more time and effort on a task and have increased exploratory behavior and greater acceptance of information 
technology when tasks create a high level of intrinsic motivation (Igbaria, Parasuraman, &Baroudi, 1996). In the past 
decades, strong commitment has been made to high quality e-assessment too, and good initial progress has been 
made. Topping (1998) reviewed the impact of peer assessment between students in higher education on writing, and 
found large positive effects. These studies provide strong evidence that good assessment practices produce large 
performance gains. In Australia, Barnes, Clarke and Stevens (2000) traced the effects of changing a high-stakes 
assessment on classroom practice, and claimed evidence for a direct causal link. An extensive literature review by 
Black and Wiliam (2002) showed that well designed formative assessment is associated with major gains in student 
attainment on a wide range of conventional measures of attainment. This result was found across all ages and all 
subject disciplines. K.H. Wang, T.H. Wang, W.L. Wang and S.C. Huang (2006) have elaborately explained the 
formative assessment strategies used in a web-based learning environment. Their research showed that both learning 
style and formative assessment strategy are significant factors affecting students’ achievement in a web-based 
learning environment. In 2008, Popham defined formative assessment as a planned process during which the teacher 
or students use assessment -based evidence to adjust ongoing learning and instruction. 

3. Method 

3.1 Formative assessment 

The notion of Formative Evaluation was first put forward by Scriven. In his classic paper “The Methodology of 
Evaluation” (1967), he introduces the terms “formative assessment” and “summative assessment”. In 1970s, Bloom 
classified assessment into diagnostic assessment, formative assessment and summative assessment. Table 1 will give 
us a clear understanding of the differences among the three assessments. 

Table 1. Three assessments classified by Bloom 



evaluation period 

evaluation aim 


(the development period of 

an event) 

Diagnostic 



assessment 

before 

earn the conditions, the starting point and diagnose 
the reasons 

Formative 



assessment 

during 

monitor the development, collect the feedback 
information and offer the improving suggestions 

Summative assessment after 

evaluate the whole effect, offer the evaluation achievement and 
make the all-round assessment 


A. Why is Formative assessment? 

From table 1, we can easily find that formative assessment focus on the process of an event, laying more emphasis 
on monitoring, correcting and improving. It is a fact that online learners are quite different from regular school 
students. They are differing in learning styles, skills, and aptitude. Their age span is big. How to monitor the online 
learners development, get updated feedback and make correspondence improvement become so urgent and 
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significant. 

B. What is the Aim of formative assessment? 

• Online instructors may take advantage of the feedback information they collected to modify their teaching 
plan, methods and process in order to improve their teaching proficiency. In the long run, they can create 
personalized learning environment to accommodate differences in online learning. 

• Online learners may use the feedback information to adjust their learning strategies, process, activities and 
etc. By realizing their own limits in learning habits or methods, they can timely adjust their studying 
methods in order to fully develop their potential intelligence and completely carry out the educational goals. 

3.2 A study report of an Interpretation online course 

A. Research background: Since the formative assessment is required and advocated by the college English 
Curriculum Requirements in China, this interpretation online course was launched to collect data for future 
establishment of assessment index system in English Language & Literature discipline. 

B: Course description: Interpretation course is a required course for the third year English majors with 2 credits. As 
part of this course, the 40 hours (32 hours online learning + 8 hours online internship) online course accounts for 1 
credit. 

Study time is recorded automatically online. The instructor is responsible for the update of the content every two 
weeks and uploads the live materials at time needed. 

C: Credit Requirements: 

* Minimum study time limit: 32 hours 

* the lowest right percentage: the instructor may set up the lowest right percentage, let’s say 55%, that is to say, if a 
learner’s score of a quiz or the right answers for an exercise is less than 55, he could not go on his study. 

* at least 5 hours peer learning 

* at least 2 hours internship 
D: Subjects 

Table 2. Participants of the Interpretation online course 


Class No. 

Students Registered 

Account 

Credit 

Class 1 

20 

20070901** 

1 

Class 2 

22 

20070902** 

1 

Class 3 

23 

20070903** 

1 


E: Course design 
(a) Self-learning: 


* Textbook learning 

* Game-based learning workshop 

(b) Group work: 

When planned and managed well, group work is an important factor in student’s success in online learning. Group 
work with a valid purpose, when thoughtfully managed with explicit rubrics and making guidelines can provide 
many benefits to students. 

* Discussion 

* Project 

(c) Role play and simulation: 

Role play and simulation are forms of experiential learning (Russell & Shepherd, 2010). Learners take on different 
roles, assuming a profile of a character or personality, and interact and participate in diverse and complex learning 


Published by Sciedu Press 


209 


ISSN 1927-6044 E-ISSN1927-6052 




www.sciedu.ca/ijhe 


International Journal of Higher Education 


Vol. 4, No. 2; 2015 


settings. Learners learn through their explorations and the viewpoints of the character or personality they are 
articulating in the environment. 

(d) Online internship 

It is based on the philosophy that learners learn best through active engagement in meaningful activities. Connecting 
activities to integrate school-and work-based learning are an essential part of School-to-Work (STW). At LaGuardia, 
the required co-op seminar helps students make connections by giving them a structure within which to reinforce 
employability skills, examine larger issues about work and society, and undertake the crucial activities of critical 
reflection.! Grubb & Badway 1998) 

Thus, like STW, the online Co-op Community model should include web-based and work-based learning such as 
seminars and online instructor-coordinator work site visits. These activities help online learners explicitly connect 
work and online learning. 

* What happened in China?-CCTV Program (Face to face) 

* What happened on campus? —Seminar; signing ceremony 
F: Evaluation of the students’ performance 

Table 3. Study report of the Interpretation online course 



Class 1 

Class 2 

Class 3 

Students registered 

20 

22 

23 

Students being interviewed 

3 

12 

4 

Total hours online 

611 

480 

706 

Group work time 

180 

87 

199 

Online internship involvement 
(Percentage) 

120 

64 

142 

Online credit owners 

15 

7 

17 

Final exam score A B 

2 5 

Job tracking 

CDF 

5 3 5 

A B C D F 

1 4 2 15 

A B C D F 

7 7 2 1 6 


G: The formula to calculate learners’ capacities’ change 

Based on the data collected in table 3, further analysis has been studied by means of using formula to calculate 
learners’ capacities’ change in the whole process of learning. 

Suppose XI stands for learners’ involvement in this online course. XI can be easily and clearly reflected in table 3 
such as total hours online, group work time, and online internship involvement, etc. 

Suppose the degree of learners’ involvement is X2. X2 shows how much the learners are engaged in the course. 

To indicate how learners’ involvement influences their capacity change, C and Cl is introduced. Cl is used to stand 
for the learners’ achievement before participating in this assessment learning while C stands for learners’ capacity 
after it. 

Suppose: 

C= f (XI, X2) + Cl (1) 

In formula (1), f (XI, X2) stands for the increasing capacity during the involvement. If a learner hasn’t taken part in 
the assessment, f (0, 0) =0. If a learner has completely devoted himself to the process, then, 

f (1,1)=1. So the learners’ capacities’ change is f (XI, X2), 0<f (XI, X2)<1 . 
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4 . Results 




4.1 A practical assessment index system 

Table 4. An effective online assessment index system 




The Evaluation Index Score 

The Index Weight 

The Final Score 

Self-learning 

Online time 

Learning contents 

The right percentage of exercise 



Peer-learning 

Discussion 




Project 



The online Co-op 

Seminars 



community 

Assessment 

Job-shadowing 

Self assessment 

Peer assessment 



Performance 

Instructor assessment 

Quiz 

Exam 



Total 





4.2 Examine the learning effect of the formative assessment index system 


A. Since the online formative assessment index system can track down, record and check the whole progress of 
learning, it can be used to calculate online learner’s capacities’ change. 

* The degree of learners’ involvement: it can be reflected from Table 3 such as online time, the right percentage of 
exercise, discussion, role play etc. 

* Interview those learners who really need help: based on the timely feedback of the system, the instructor can have a 
clear idea about the learners’ involvement which will directly influence their future achievement. The instructor 
should interview those learners who really need help, or talk about the problem in the Group Work (discussion) 
section and offer the best solutions. 

B. Since the online formative assessment index system can collect statistics on what materials or content the learners 
show interest in, it can be used to create a personalized learning environment. 

* The characteristics and preference of the learners: The data collected from Table 3 such as online time, the learning 
contents, co-op etc. can indicate the learner’s preference 

* Design an appropriate learning environment based on learners’ preferences or differences (Sampson, 
Karagiannidis , & Kinshuk , 2002). Acceptance of constructivist approaches in mainstream education has brought 
about the realization that learning process are more effective and successful when the instructional approaches and 
content are geared towards individual learners.(Kinshuk, 2012)Unlike traditional learning environment which is 
often designed for the “average learner”, the online personalized learning environment will provide the materials that 
are depend on the learner’s characteristics such as learning styles, skills, interests etc. 

C. Since the online formative assessment index system creates a unique online Co-op community, it can be used as a 
new STW model. 

* The participation and performance of the learners in online collaborative learning activities: The number of 
learners who participate in the online internship shows learners enthusiasm to know the external world. The more 
online hours the learners engaged in, the more popular the courses become. 

* Provide a full Toolkit for the future job market: It is about giving learners the full set of skills they need to enter 
into a career. The instructor should design practical and attractive online internship course such as a Live Interview, a 
Videotaped Job shadowing or even great talks with local entrepreneurs etc. 
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5. Discussion 

With advancements in communication technology, online education may one day be able to replicate even the most 
technical of school work, including labs and workshops. There are even big possibilities around simulations and 3D 
virtual environment. And it is only going to continue to grow. This study indicated that it is important to try a new 
STW model (it is called online Co-op community )as a typical formative assessment index. The theoretical approach 
of communities of practice is characterized by mutual engagement, joint enterprise, shared repertoire, and negotiated 
meaning (Alavi & Leidner, 2001). Under this structure, the community creates the social fabric of learning. 
Therefore, this kind of learning is in terms of changing forms of participation in ongoing cultural activities (Saade & 
Bahli, 2005;Legris , Ingham , & Collerette ,2003). 

This study was motivated by the need to develop an effective online learning. This study utilized the theory of 
formative assessment, the peer learning idea and take advantage of the theory of community of practice in an attempt 
to identify opportunity, suggest ways forward and create a scientific and effective formative assessment index system 
of online learning. Thus, this paper established a comprehensive, objective, scientific and accurate formative 
assessment index system to ensure the online learning would be mostly successful. 

Although the new concept is theoretically practical, the design and assessment of the actual Co-op community is still 
a big challenge and major challenges ‘going to scale’ yet to be faced. Cultural differences should be considered. A 
great deal of innovative work is needed, coupled with a grounded approach to system-wide implementation. 
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